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Abstract 

The use of data-based decision making (DBDM) in schools to drive educational improvement and success has been strongly 
promoted by educational experts and policymakers, yet very little is documented about the actual DBDM practices used 
in schools. This study examines DBDM practices in 25 middle schools through 80 standardized observations of data team 
meetings and through survey responses. DBDM practices in terms of structure, process, content, quality, and alignment 
between self-report and observation are described. Key findings include the following: (a) The average amount of time 
spent discussing an individual student was brief, less than 2 min; (b) on average, actionable decisions following discussions 
of behavior or reading issues were made 34% to 40% of the time; and (c) there was weak alignment on key topics between 
self-reported and observed DBDM practices. These findings underscore the need for additional studies of DBDM practices 
in school and of the impact of DBDM practices on important student outcomes. 
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In the past decade, the expectation that educators use data to 
inform instructional decisions has increased dramatically, as 
has awareness of the potential that this process, known as 
data-based decision making (DBDM), holds for improving 
student outcomes (Mandinach, 2012). This combination has 
led to more widespread adoption of DBDM practices in 
schools, across all levels of educational delivery. The types 
of data collected and the ways in which data are used vary, 
depending on the role and purpose of the individual or team 
using the data. The literature on formative assessment, 
including curriculum-based measurement (CBM), has con¬ 
tributed to the trend toward increased data use in schools. 
This literature demonstrates that regular, reliable, and valid 
evaluation of essential skills paired with instructional modi¬ 
fications, as necessary, can have a positive and significant 
impact on student achievement (see, for example, Fuchs & 
Fuchs, 1986; Marcotte & Flintze, 2009). Stan Deno and col¬ 
leagues (e.g., Deno, 1985) pioneered much of the early work 
on collecting and integrating frequent, current assessment 
data to identify and address skill deficits in critical academic 
areas. Their groundbreaking work in CBM is considered by 
many to be the original model of DBDM in education. 

A critical impetus for the increased use of data in schools 
was the No Child Left Behind Act (NCLB; 2001), which 


mandated that schools and states systematically collect, 
report, and respond to data measuring student achievement 
in key content areas. Because NCLB also legislated sanc¬ 
tions for schools whose achievement data demonstrated 
inadequate progress toward grade-level standards, schools 
and districts were strongly incentivized to collect data and 
use them to improve student achievement, at least on state¬ 
wide assessment measures. What NCLB did not do, how¬ 
ever, was to specify the mechanisms for schools and 
teachers to effectively implement and evaluate DBDM 
practices (Wayman & Stringfield, 2006). Thus, although 
teachers and administrators typically have access to multi¬ 
ple sources of student performance data, relatively little is 
known about the specific DBDM practices that school per¬ 
sonnel utilize to improve student achievement. This study 
begins to fill that need by describing and evaluating DBDM 
practices in 25 middle schools, using both standardized 
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observations of data team meetings and responses from a 
survey on school DBDM practices. 

Current Recommendations for Best 
Practices in DBDM 

Despite dramatic increases in access to data, few educators 
have been adequately prepared to efficiently and effectively 
navigate the information available to them (Mandinach, 
2012), resulting in a substantial discrepancy between the 
amount of student data accumulated and the corresponding 
knowledge of how to use those data to improve outcomes 
for students. Recently, researchers have begun working to 
bridge this gap by developing and disseminating recom¬ 
mendations for how districts, schools, and teams, across all 
grade levels, should use data (e.g., Hamilton et al., 2009; 
Kekahio & Baker, 2013). For example, the National Center 
on Intensive Intervention (NCII, www.intensiveinterven- 
tion.org) offers multiple resources and implementation and 
instructional support on using DBDM to deliver intensive 
and individualized interventions to students with significant 
behavioral and learning needs. These recommendations all 
offer similar definitions of DBDM and share a number of 
common themes. 

One common theme is that effective DBDM processes 
are based on an iterative cycle of data collection, analysis, 
decision making, and follow-up (Hamilton et al., 2009; 
Mandinach & Jackson, 2012). Mandinach, Honey, Light, 
and Brunner (2008) have developed and popularized a data 
use framework that elucidates how raw data can be trans¬ 
formed into useable knowledge and data-driven actions. 
Within their framework, raw data are collected from a num¬ 
ber of available sources and organized so that users (e.g., 
teachers, administrators, specialists, etc.) can make sense of 
the data. In the next level, users analyze patterns and trends 
in the data and summarize their findings to explain how a 
specific student or group of students is performing, typi¬ 
cally in relation to pre-established criteria or expectations. 
Finally, summarized data are transformed into actionable 
knowledge. Data users synthesize the information they have 
generated so far and prioritize what actions to take first. 
Identified actions should be implemented, and the effect of 
those actions should be monitored and evaluated. These 
new data can then serve as the foundation for another itera¬ 
tive cycle of DBDM. 

Other common themes regarding best practices for 
DBDM include recommendations that instructional deci¬ 
sions be based on multiple sources of (reliable and valid) 
data, and that the DBDM process incorporate effective pro¬ 
fessional development regarding understanding and using 
data, an infrastructure of support to build and maintain 
capacity, and the use of data teaming, rather than relying on 
a single, qualified individual to drive the DBDM process 
(Hamilton et al., 2009; Kekahio & Baker, 2013; Mandinach, 


2012). Data teaming provides a unique opportunity for 
teachers, administrators, and related service personnel to 
collaboratively discuss the needs and progress of students 
in the context of classroom or school issues, such as 
response to instructional groupings based on statewide 
assessment results or alignment of instruction within and 
across grade levels. Furthermore, data teaming allows staff 
without data analysis skills to learn how to collect, analyze, 
and apply data from more experienced colleagues or data 
coaches (Means, Padilla, & Gallagher, 2010). 

Current Understanding of Data 
Teaming Practices in Schools 

Over the years and in response to different legislation or 
priorities, schools have used the concept of teaming to pro¬ 
vide support to students and teachers. Problem-solving 
teams have long served multiple purposes, including diag¬ 
nostic assessment, teacher support, and identification of 
behavioral and academic challenges (Bahr & Kovaleski, 
2006). In addition, more recent legislation (Individuals 
With Disabilities Education Act [IDEA], 2004) encouraged 
schools to adopt a team-based multi-tiered Response to 
Intervention (Rtl) model for all students. 

Nevertheless, direct evidence of how these school- 
based teams access, review, and use data is limited. Most 
research on data use in schools relies primarily on self- 
report, focus groups interviews, or surveys, rather than on 
standardized, direct observations of DBDM in practice. 
One exception comes from the positive behavioral and 
interventions support (PBIS) literature. Newton, Horner, 
Todd, Algozzine, and Algozzine (2012) conducted obser¬ 
vations of PBIS teams to document fidelity of implemen¬ 
tation to their specific data teaming process, called the 
Team-Initiated Problem Solving (TIPS) model. Four ele¬ 
mentary school data teams in two states received profes¬ 
sional development at a workshop to implement the TIPS 
model. The researchers then used a standardized observa¬ 
tion protocol to document adherence to the TIPS model, 
and found that data teams were in fact able to implement 
the TIPS model with fidelity. Although this study is 
encouraging in that it demonstrates that typical school 
staff can effectively implement a DBDM process when 
provided with adequate professional development and 
coaching, it does not answer the question of how typical 
data teams examine and use data in the absence of a speci¬ 
fied framework and associated professional development. 
Findings from other studies including observations of data 
team meetings, in elementary, middle, or high school, are 
primarily reported as vignettes or case descriptions (e.g., 
Feldman & Tung, 2001; Marsh, Farrell, & Bertrand, 2014), 
rather than quantitative summaries of structured observa¬ 
tions. The present study begins to address this gap in the 
literature. 
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Middle School Intervention Project 

The data presented here were collected as part of the Middle 
School Intervention Project (MSIP), an effort to evaluate 
the impact of a multi-component intervention on improving 
academic and behavioral outcomes among middle school 
students with low reading performance. We worked with 
middle school teams because the participating districts had 
identified struggling middle school readers as one of their 
top priorities for schools’ improvement. At the outset of 
MSIP, each participating school district had systems in 
place for examining student data and agreed to integrate 
regular review of academic and behavioral data for MSIP 
students into each school’s data team practices. We con¬ 
ducted multiple, direct observations of data team meetings 
over the course of the school year, providing an opportunity 
to examine how data teams operate in typical middle school 
settings. 

Research Questions 

We addressed the following research questions: 

Research Question 1: How is DBDM implemented in 
the middle schools in the sample? 

We describe the structure, process, content, and decisions 
made in the observed data team meetings. 

Research Question 2: To what extent do self-reported 
data team practices align with observed data team 
practices? 

We examine the alignment between responses on a stan¬ 
dardized survey of school practices with the findings from 
multiple, standardized observations of data team meetings 
at each participating school. 

Method 

In this article, we report on findings from the third year 
(2012-2013) of a 5-year longitudinal study, when the stu¬ 
dent sample was in eighth grade. Participating schools uti¬ 
lized a three-pronged intervention to support their at-risk, 
struggling readers: (a) a reading intervention component 
that provided targeted reading support to struggling readers, 
(b) a school engagement intervention component to 
strengthen students’ behavioral and psychological engage¬ 
ment with school, and (c) a DBDM team component. 
Reading interventions were typically offered as a reading 
class. Curricula, teacher to student ratio, and duration of the 
intervention varied across districts and schools. School 
engagement interventions varied widely across schools, 
including, for example, extracurricular clubs, tutoring, and 


behavior-based check-in/check-out programs. In the DBDM 
component, data teams met with the intention of reporting 
academic achievement and student engagement data for 
intervention students, using those data to evaluate the effec¬ 
tiveness of the interventions for a particular student or 
group of students, and modifying interventions as necessary 
to improve outcomes. 

Setting 

This study was conducted in 25 middle schools within six 
school districts in the Pacific Northwest. One of the six dis¬ 
tricts (four schools) began participation in the second year 
of the study. Districts ranged in size from 5,865 to 39,488 
students, with 348 to 1,064 students in each middle school. 
Within the districts, 36.2% to 63.2% of students were eli¬ 
gible for Free or Reduced Lunch, 29.30% to 48.4% were 
minority students, and 11% to 26% of students were English 
learners. 

Participants 

DBDM teams. In each year of the study, we observed 
DBDM team meetings in the fall, winter, and spring. 
Schools held a variety of teacher or staff-led meetings each 
week. To identify which meetings to observe, school staff, 
in collaboration with a district-based liaison to the project, 
identified which of those meetings focused on DBDM for 
eighth grade students. In some schools, these meetings 
focused only on students in the MSIP intervention. In other 
schools, these meetings included discussion of both MSIP 
and non-MSIP intervention students. DBDM teams were 
observed for the duration of each meeting that was focused 
on discussion of student and/or group data. 

Survey respondents. The staff member most familiar with 
the school’s DBDM practices was asked to complete the 
DBDM portion of the School Interventions Survey (SIS; 
Crone, Smith, et al., 2010), described below. Respon¬ 
dents self-identified as principals (n = 8, 32%), assistant 
principals (n = 5, 20%), literacy coaches or reading spe¬ 
cialists (/? = 5, 20%), school counselors (« = 3, 12%), 
classroom teachers (n = 2, 8%), and Rtl or intervention 
specialists (n = 2, 8%). 

Measures 

S/S. The SIS is a 45-item questionnaire designed by the 
research team to collect self-reported data regarding each 
school’s practices on reading interventions, student engage¬ 
ment interventions, and DBDM teams. The survey consists 
of both multiple-choice and open-response questions. 
Respondents completed the survey using Qualtrics (2015), 
a secure, online survey collection tool. 
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Data Team Tool for Observation (DTTO). The DTTO is an 
observation protocol developed during the first year of 
MSIP to capture the content and quality of data team meet¬ 
ings. Revisions to the protocol occurred at the end of each 
academic year, resulting in three slightly different versions 
of the tool (DTTO 1.0, 2.0, and 3.0). DTTO 3.0 was used 
for the eighth grade year of the study. A copy of the DTTO 
3.0 and corresponding manual (Crone, Nelson-Walker, 
Cohen, Coughlin, & Schwartz, 2012) are available from the 
first author. The DTTO 3.0 consists of four sections, 
described below. 

Data team cover sheet. Observers use the cover sheet to 
document background information (e.g., date, school, team 
name, team purpose) and structural features of the data 
team (e.g., start and end times, number of team members 
present, whether or not an agenda was used). This part of 
the protocol also requires the observer to meet with the data 
team leader immediately following the meeting to answer 
questions about topics that cannot be ascertained during the 
meetings (e.g., follow-up behaviors to support plan imple¬ 
mentation). 

Data team participant list. The data team participant list 
is used to document meeting participants and their roles on 
the data team and in the school. 

Students reviewed summary sheet (SRSS). Observers use 
the SRSS to document information regarding data team dis¬ 
cussions about individual students and groups of students. 
For each separate discussion, problem areas (e.g., academic, 
behavioral), quantitative and qualitative data sources refer¬ 
enced, and action plans are recorded. Examples of quanti¬ 
tative data sources include state test scores for reading or 
math, attendance records, and scores on other reading tests. 
Examples of qualitative data sources include relevant dis¬ 
cussions of the child’s home life, discussions of the child’s 
peers, and work samples. Detailed definitions of each data 
source are provided in the DTTO 3.0 manual, available 
from the first author. 

The SRSS also allows observers to indicate, within the 
discussion of a particular student or group, whether any 
actions were considered and/or adopted. Observers indi¬ 
cated “decision made” when a specific action to be taken 
was discussed or written down. Sometimes, the decision 
made was to maintain the student in the current 
intervention(s). Observers marked “goal set” when the team 
enumerated the expected outcomes for the decision made 
within a given content area. “Timeline established” was 
selected if the team indicated when outcomes were expected 
and when they would be reviewed, given any decision(s) 
made. “Person assigned” was selected when the team desig¬ 
nated an individual to facilitate implementation and dis¬ 
seminate information for any decisions made. 


Quality of data team structure and process. At the con¬ 
clusion of each data team meeting, observers completed a 
quality ratings form consisting of 29 items divided into four 
sections—structure, process, student-level discussions, and 
group-level discussions. Observers rated data team struc¬ 
tures and processes using a 3-point scale: 0 = element absent 
or not observed, 1 = inconsistent level of implementation, 2 
= high level of implementation. 

Procedures 

Calculating inter-observer agreement (IOA). Members of the 
MSIP research team conducted all data team observations. 
After training on the DTTO 3.0, paired observations were 
conducted until IOA criteria were met in three areas: (a) the 
number of students and groups identified, (b) total score on 
the Quality of Data Team Structure and Process sheet, and 
(c) the information captured for each sub-category and total 
on the SRSS. IOA for the number of students or groups 
observed was calculated by dividing the total number of 
agreements by the total number observed by the primary 
observer. IOA criteria for students or groups were set at 
90% or greater agreement. 

IOA for the quality ratings sheet was calculated by 
obtaining the difference in total score between primary and 
secondary observer and dividing by the total number of 
points possible. This number was subtracted from 1 and 
multiplied by 100 to obtain a total percent agreement. The 
IOA criterion for the quality ratings page was set at 90% or 
greater on the total score. 

IOA for the SSRS was calculated as follows: An agree¬ 
ment on the SRSS was defined as any time both observers 
endorsed (fill in a bubble) or both observers did not endorse 
(leave a bubble blank) a given item on the sheet. In contrast, 
a disagreement on the SRSS occurred any time one observer 
endorsed an item, while the other observer did not endorse 
that item. The total number of agreements was tallied for 
the entire observation, as well as the total for each of the 
following sub-categories: (a) areas discussed, (b) nature of 
quantitative data, (c) nature of qualitative data, and (d) 
action plan. The total number of agreements for each cate¬ 
gory was divided by the total number of possible items 
within that category multiplied by the number of rows of 
data completed. This number is multiplied by 100 to obtain 
a percent agreement for each category as well as for the 
overall total. IOA criteria for the SRSS were set at 85% or 
greater for each sub-category, and an average of 90% or 
greater across all sub-categories. 

Observation sampling plan. Observers completed three sepa¬ 
rate rounds of data team observations, observing a total of 
80 distinct data team meetings across the six participating 
districts. In 25% of the observations, two observers were 
paired for purposes of computing IOA. Average IOA for the 
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number of students/groups was 97.1%; average IOA for 
total score on the quality ratings was 95.5%; and average 
IOA for the overall total on the SSRS was 97.9%. 

Data teams at each school were observed between 2 and 
7 times throughout the school year. Across districts, 19 
observations were conducted in the fall, 33 were conducted 
in the winter, and 28 were conducted in the spring. Three or 
more observations per school were achieved at 80% of 
schools (20 schools); 5 schools were observed only twice, 
either because (a) those data teams met fewer than 3 times 
during the year, or (b) the first opportunity to observe the 
team at the beginning of the school year occurred before 
observations began. 

Results 

How Is DBDM Implemented in the Middle 
Schools in the Sample? 

Structure of data team meetings. On average, 7 data team 
members (SD = 2.0) attended each observed DBDM meet¬ 
ing, with a range of 2 to 12 participants. At least one admin¬ 
istrator attended 78.4% of the meetings, a special education 
instructor attended 68.9% of the meetings, and reading 
intervention and English language learner (ELL) specialists 
attended 54.1% and 35.1% of the meetings, respectively. 
The most frequent participants on DBDM teams were 
grade-level teachers, mostly in language arts and math con¬ 
tent areas, who were present at 74.3% of meetings, and 
made up 40.7% of all participants. The number of teachers 
per meeting ranged from 0 to 12. 

During observed data team meetings, an agenda (written 
or verbally delivered) was used 96.2% of the time, and 
notes were taken in 92.0% of meetings. Data were typically 
visually displayed in some format (67.53% of observa¬ 
tions), and most teams were able to readily access electronic 
records during the meeting (67.11%). Some teams used a 
participant rotation approach, so that key individuals (e.g., 
the teacher for a specific subject) attended the meeting only 
to discuss specific students. Participant rotation was used in 
32% of the meetings observed. 

Process characteristics of data team meetings. The frequency 
and duration of DBDM meetings to discuss MS1P students 
varied within and across districts. When asked about meet¬ 
ing frequency on the SIS, the most common response (n = 
8) was “once per month.” Two schools indicated that data 
teams met weekly. The average duration of observed data 
team meetings, by school, was 49.5 min (SD = 18.47). 

At the conclusion of each data team meeting, observers 
asked data team leaders a standard set of questions about 
non-observable aspects of the DBDM process, including 
whether and how data were shared with key stakeholders. 
Most data team leaders reported that team members were 


asked in advance to bring data to the meetings (76% of 
observations), and in most cases, agenda items, such as a 
list of the students to be discussed, were distributed prior to 
the meeting (87.5%). Team leaders further indicated that 
decisions made in the meeting were typically shared with 
important stakeholders: 91% of teams provided summaries 
of meeting notes to team members, and nearly all reported 
sharing their rationale with the teachers (93.6%), students 
(84.4%), and parents (82.3%) likely to be affected by deci¬ 
sions or action plans. 

Data team follow-up. The extent to which data teams 
followed up on decisions made during the data team meet¬ 
ings could not be assessed directly through observation. 
Thus, here we report on SIS responses to related questions. 
Schools were asked, “Does the school collect data and/or 
conduct observations to assess if the reading intervention is 
being implemented the way that it is intended by the pub¬ 
lishers of the program?” Only four schools (16%) indicated 
this was “firmly in place,” and another four indicated that 
this was “partially in place.” The remaining 17 schools indi¬ 
cated either that they had a plan to put this in place in the 
coming school year, or were not aware of any plans to put 
this in place. Encouragingly, however, of the eight schools 
that responded “firmly in place” or “partially in place” to 
the previous question, six then responded to the follow-up 
question, “Does the team use data regarding the imple¬ 
mentation of the published program to make modifications 
to the implementation of the reading intervention?” with 
“firmly in place.” The remaining two schools responded 
with “partially in place.” 

Content of data team meetings. The DTTO 3.0 captures 
details about the content of the discussions in which data 
teams engage. A discussion was defined as any time one or 
more team members talked about an individual student or 
particular group. This might involve just a brief presenta¬ 
tion of a test score or a lengthy conversation about a stu¬ 
dent’s performance. A discussion ended as soon as the focus 
of the discussion switched to a different student or a differ¬ 
ent group. In some meetings, the team had multiple discus¬ 
sions about the same individual or group. We organize our 
description of the content into three areas: (a) the extent to 
which data teams focused on students and groups, (b) the 
types of data sources discussed, and (c) the frequency with 
which actionable decisions were made. 

Focus on students and groups. The number of discussions 
about students or groups at each meeting varied dramati¬ 
cally across schools. Some data teams chose to focus an 
entire meeting on just a few students, whereas others used 
the meetings to briefly review standardized assessment 
data for a large number of individual students identified as 
struggling readers. Although most data team discussions 
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focused on individual students (92.6%), some data teams 
also discussed groups of students. For example, a team 
might discuss the progress of an entire reading intervention 
class and make decisions about modifications to interven¬ 
tions at the group, rather than student level. Across all meet¬ 
ings observed, the number of discussions of groups ranged 
from 0 to 18 (M = 4.4, SD = 5.4). Across all meetings, the 
number of discussions about students ranged from 0 to 147. 
Multiple discussions of the same student or group within or 
across meetings were treated as separate discussions in the 
DTTO protocol. Thus, meetings with a high number of dis¬ 
cussions focused on multiple students, and often returned to 
discuss the same individuals on multiple occasions within 
the meeting. Averaged across schools, the mean number of 
discussions about students per meeting was 18.2 (SD = 6.8). 
Some students were discussed at every observed meeting 
for a given school. 

The average length of time spent per discussion also var¬ 
ied dramatically across schools. Averaged across schools, 
discussions of individual students ranged in length from 53 
s to 7 min, 35 s long (M = 2 min, 29 s; SD = 1 min, 39 s). 
Discussions of groups ranged in length from 1 min, 35 s to 
47 min, 3 s (M = 7 min, 24 s; SD = 11 min, 24 s). The aver¬ 
age number of discussions about students and groups and 
the corresponding average amount of time spent for each 
discussion are summarized, by school, in Table 1. The per¬ 
cent of discussions devoted specifically to MSIP interven¬ 
tion students (out of all the students discussed) is also 
presented. On average, almost half of all student discus¬ 
sions (47.5%) were focused on MSIP intervention students, 
although they only represented about a quarter of the stu¬ 
dent population. However, the mean length of discussion 
for intervention students (M = 1 min, 37 s; SD = 2 min, 43 
s) was even shorter than the mean for all students. 

Actionable decisions. The extent to which discussions of 
students or groups resulted in an actual decision made and 
action taken (i.e., goal set, timeline established, and/or per¬ 
son assigned) is presented in Table 2. Data are presented for 
each school, across all observations for that school. Schools 
displayed substantial variability in the extent to which data 
teams concluded a discussion of a student or group by mak¬ 
ing a decision on what to do next. The percent of discus¬ 
sions that led to decisions made ranged from 0 to 92. Given 
that a primary objective of DBDM is decision making, it is 
instructive to find such an enormous range in the amount of 
actual decision making that occurs. This finding indicates 
that in some data team meetings, there were no decisions 
made whatsoever, whereas in others, almost every discus¬ 
sion of a student or group ended with some type of decision 
made. On average, 40% of discussions of reading issues 
resulted in a decision (SD = 24%), whereas 34% of discus¬ 
sions of behavior or school engagement issues resulted in a 
decision (SD = 26%). 


Data sources. The SIS asks respondents to indicate which 
data sources are regularly reviewed at the data team meet¬ 
ings. Scores on standardized state assessments and atten¬ 
dance data were the data sources most frequently endorsed 
(by 80% of schools). Respondents further indicated that 
grades (72% of schools) and anecdotal reports about daily 
functioning in class (76%) were important data sources for 
the DBDM teams. 

These SIS self-reports were compared with the DTTO pro¬ 
tocol, on which observers recorded when quantitative or qual¬ 
itative data were discussed in regard to a specific student or 
group and indicated the specific data source used. In discus¬ 
sions of individual students, the most frequently used data 
source was quantitative data on students’ test scores. 
Performance on the state standardized reading test was refer¬ 
enced in 35.0% of student discussions, and other quantitative 
reading scores were referenced in 36.9% of student discus¬ 
sions. Work samples were referenced in only 1.0% of student 
discussions. In discussions regarding student behavioral 
issues, quantitative data were referenced in only 6.14% of dis¬ 
cussions, whereas qualitative data were referenced in 31.8% 
of discussions. Attendance data were referenced in about a 
tenth of all student discussions ( quantitative = 10.3%; qualita¬ 
tive = 10.5%). During group discussions of reading, qualita¬ 
tive data were referenced more frequently (35.5%) than 
quantitative data (19.1% to 23.6%). Additional details regard¬ 
ing the observed use of data sources are provided in Table 3. 

Quality ratings. The quality of the team meetings as rated by 
observers is summarized as a percentage of total possible 
points received for each of the four sections. In general, 
observers rated teams high on items related to data team 
structure (M= 84.5%; SD = 17.7%) and data team process 
(M= 79.5%; SD = 16.6%). Data teams were rated lower on 
student-level (M = 64.0%; SD = 23.1%) and group-level 
items (M= 52.8%; SD = 23.8%), but still received a major¬ 
ity of the possible points. 

To What Extent Do Self-Reported Data Align 
With Observed Data Team Practices? 

Our second research question was to determine the extent to 
which schools’ self-report of data team practices aligned 
with what was actually observed in data team meetings. To 
answer this question, selected questions from the SIS were 
compared with the data team observations. Results of these 
comparisons are reported in Tables 4 and 5. 

On the SIS, respondents were asked, “Are there systems in 
place to collect and manage data regarding the academic 
achievement of all the students in your school,” and “Is there a 
systematic process in place for reviewing data and making 
decisions about the progress of individual students?” 
Respondents could answer “yes, firmly in place,” “partially in 
place,” or “no, not in place,” or indicate that there was nothing 



Table I. Mean Number of Discussions and Duration of Discussions for Students and Groups. 
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All schools 80 18.2 1,368 0 82 6.8 0:02:29 0:01:39 650 47.5 NO 0:07:24 0:11:24 

(n = 25) 
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Table 2. School Action Plans for Reading Intervention and Student Engagement. 


Reading intervention Engagement issues 


School ID 

Discussions 

Decisions 

made 

Actions 

taken 

% discussions that 
led to decisions 

Discussions 

Decisions 

made 

Actions 

taken 

% discussions that 
led to decisions 

A 

56 

28 

13 

50 

15 

10 

2 

67 

B 

15 

6 

5 

40 

25 

2 

1 

8 

C 

130 

65 

63 

50 

52 

2 

2 

4 

D 

138 

99 

52 

72 

33 

2 

2 

6 

E 

13 

4 

3 

31 

59 

16 

14 

27 

F 

2 

0 

0 

0 

42 

20 

13 

48 

G 

20 

7 

2 

35 

46 

28 

23 

61 

H 

77 

39 

37 

51 

53 

17 

12 

32 

1 

46 

29 

25 

63 

12 

1 

0 

8 

J 

7 

6 

6 

86 

0 

0 

0 

0 

K 

35 

22 

15 

63 

27 

21 

19 

78 

L 

5 

2 

2 

40 

38 

17 

16 

45 

M 

13 

9 

5 

69 

17 

1 1 

9 

65 

N 

0 

0 

0 

0 

14 

5 

4 

36 

O 

6 

3 

1 

50 

16 

2 

0 

13 

P 

5 

1 

1 

20 

4 

1 

1 

25 

Q 

16 

6 

6 

38 

1 1 

2 

1 

18 

R 

37 

6 

2 

16 

44 

17 

14 

39 

S 

34 

6 

5 

18 

8 

6 

6 

75 

T 

128 

31 

30 

24 

58 

3 

3 

5 

U 

54 

12 

9 

22 

71 

15 

4 

21 

V 

7 

3 

1 

43 

12 

1 1 

9 

92 

w 

45 

18 

18 

40 

23 

10 

10 

43 

X 

38 

31 

28 

82 

28 

3 

3 

1 1 

Y 

56 

4 

1 

7 

38 

1 1 

8 

29 


Note. Discussions, decisions made, and actions taken are all raw total counts across all meetings at a school. 


currently in place. Most schools indicated that they had sys¬ 
tems in place for collecting and managing reading intervention 
data (76%) and behavior/engagement data (68%). Conversely, 
fewer than half indicated that a systematic process for review¬ 
ing data and making decisions was firmly in place for reading 
intervention (44%) and behavior/engagement data (40%). 

Next, we examined corresponding evidence for observed 
use of quantitative and qualitative data and frequency of deci¬ 
sion making by school. We defined strong evidence as data 
sources or decision-making behavior that were observed for 
75% or more of discussions across meetings, partial evidence 
as data sources or decision-making behavior that were 
observed for 26% to 74% of the discussions, and limited evi¬ 
dence as data sources or decision-making behavior that were 
observed for 25% or less of the total number of discussions. 

When looking across these two sources of evidence (i.e., 
self-reports and data team meeting observations), we found 
that although 76% of schools reported having systems firmly 
in place for collecting and managing reading intervention data, 
DTTO observations demonstrated strong evidence of actually 
using quantitative data for only 48% of schools and qualitative 


data for only 8% of schools. With regard to engagement inter¬ 
ventions, 68% of schools reported having systems firmly in 
place for collecting engagement data, but observations showed 
that no schools demonstrated strong evidence for use of quan¬ 
titative engagement data, whereas 84% of schools demon¬ 
strated strong evidence for the use of qualitative engagement 
data. Furthermore, the discrepancy between self-report on the 
frequency with which decisions are made and the observed 
evidence for those decisions is even greater. Although 44% of 
schools reported having a systematic process firmly in place 
for making decisions regarding reading interventions, strong 
evidence was observed for only 8% of the discussions that 
concluded in a decision. Similarly, 40% of schools reported 
having a systematic process firmly in place with regard to deci¬ 
sion making for engagement/behavioral support, but strong 
evidence for this was observed for only 12% of schools. 

Discussion 

Many educational experts have advocated for the utility of 
DBDM as a mechanism for driving educational improvement 
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Table 3. Frequency Counts and Percentages for Types of Data Discussed in Data Team Meetings (n = 80). 



Student discussions (n 

= 1,368) 

Group discussions (n 

= 1 10) 

Types of data discussed 

Frequency counts 

% discussions 

Frequency counts 

% discussions 

Academic reading 

Quantitative 

State reading Test 

479 

35.0 

21 

19.1 

ORF 

301 

22.0 

21 

19.1 

All other 

505 

36.9 

26 

23.6 

Qualitative 

321 

23.5 

38 

34.5 

Other academic areas 

Quantitative 

State math test 

174 

12.7 

5 

4.5 

All other 

255 

18.6 

10 

9.1 

Qualitative 

Work sample 

14 

1.0 

6 

5.5 

All other 

323 

23.6 

28 

25.5 

Attendance 

Quantitative 

141 

10.3 

4 

3.6 

Qualitative 

144 

10.5 

3 

2.7 

Behavior 

Quantitative 

84 

6.1 

1 

0.9 

Qualitative 

Motivation 

190 

13.9 

12 

10.9 

All other 

435 

31.8 

22 

20.0 

English language learning 
Quantitative 

English lang. status score 

1 1 

0.8 

1 

0.9 

All other 

8 

0.6 

0 

0.0 

Qualitative 

77 

5.6 

1 

0.9 

Special education-related 
Quantitative 

6 

0.4 

0 

0.0 

Qualitative 

119 

8.7 

4 

3.6 

Health 

Quantitative 

0 

0.0 

0 

0.0 

Qualitative 

1 1 1 

8.1 

1 

0.9 

Other 

Qualitative 

Family 

276 

20.2 

1 

0.9 

Peer 

113 

8.3 

5 

4.5 


Note. ORF = oral reading fluency. 


(Hamilton et al., 2009; Mandinach, 2012). However, very lit¬ 
tle has been published detailing the actual DBDM practices 
used in schools, including the extent to which schools’ data 
team practices lead to data-driven decisions; affect student 
performance; or align with schools’ own perceptions of their 
DBDM practices. 

We attempted to begin addressing this practice to research 
gap by documenting the DBDM practices of 25 middle schools 
in six school districts, each of which had prioritized interven¬ 
tion for struggling readers as a key goal for school improve¬ 
ment. In this section, we focus on four findings that we believe 


warrant further consideration: (a) the amount of time spent on 
discussions of individual students or groups, (b) the extent to 
which data (quantitative and qualitative) were presented at 
DBDM meetings, (c) the frequency with which discussions led 
to actionable decisions, and (d) the degree to which self- 
reported DBDM practices aligned with observed practices. 

Time 

Given the duration of the meetings and the number of 
school staff members who participate, significant school 
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Table 4. Proportions and Frequencies 

for Self-Reported Responses Regarding Middle School Data-Based Decision Making (n : 

= 25). 

Survey focus 

Survey question 
topic 

“Yes, firmly in place" 

“Partially in 

place” 

“No, not in 
other 

i place; 

Total 

squency 

Proportion Frequency 

Proportion 

Frequency 

Proportion 

Frequency fri 

Reading 

Systems for 

.76 19 

.16 

4 

.08 

2 

25 

intervention 

collecting/managing 








intervention data 







Behavior/ 


.68 17 

.20 

5 

.08 

2 

24 

Engagement 








Reading 

Systematic process 

.44 1 1 

.36 

9 

.12 

10 

23 

intervention 

for reviewing data/ 








making decisions 







Behavior/ 


.40 10 

.32 

8 

.20 

5 

23 

engagement 








Note. “No, not in 

place; other” included the following response options: “pi 

an to put in place 

in 2012-2013” 

; “no, not aware 

of a plan to put in 


place”; “not applicable, no team that examines the data”; and “don’t know.” 

“Frequency” represents the number of schools. One to three expert 

representatives (i.e., in reading, engagement, and/or data) from each school 

were assigned to fill out the survey or sections of the survey. 



Table 5. Proportions and Frequencies 

for the Processes of Middle School Data Team Meeting Observations (n = 

25). 

Strong 

Partial 


Limited 


Observation focus Proportion 

Frequency 

Proportion 

Frequency 

Proportion 

Frequency 

Total frequency 

Reading support 

Evidence for use of data 

Quantitative .48 

12 

.40 

10 

.12 

3 

25 

Qualitative .08 

2 

.52 

13 

.40 

10 

25 

Decisions made .08 

2 

.64 

16 

.28 

7 

25 

Engagement support 

Evidence for use of data 

Quantitative .00 

0 

.40 

10 

.60 

15 

25 

Qualitative .84 

21 

.12 

3 

.04 

1 

25 

Decisions made .12 

3 

.48 

12 

.40 

10 

25 


Note. Strength of the evidence is defined as the proportion of students and groups for which a given area was discussed during an observation at each 
school. “Strong evidence” indicates that the focus was observed 75% or more of the total number of discussions had for students and groups; “partial 
evidence” indicates that the focus was observed between 26% and 74% of the total number of discussions had for students and groups; and “limited 
evidence” indicates that the focus was observed 25% or less of the total number of discussions had for students and groups. Reading data could include 
discussions around the students’ state standardized reading scores, oral reading fluency, other reading scores (e.g., CBMs), and classroom assignments 
and tests; Behavior/engagement data could include discussions around attendance, behavior reports, and motivation. “Frequency” represents the number 
of schools. CBM = curriculum-based measurement. 


resources and time are being devoted to the DBDM process. 
However, we question whether staff time and resources are 
being used efficiently and productively. Presumably, the 
students who were discussed at the observed data team 
meetings were those students who were struggling the most 
in school, either academically or behaviorally, or both. Data 
team meetings likely represent one of the few opportunities 
school personnel have to review students’ progress and per¬ 
formance data, make data-based decisions regarding inter¬ 
ventions or instructional modifications, and monitor the 
outcomes of those plans. Given the high stakes associated 
with students’ failure (e.g., Finn & Owings, 2006; Harlow, 
2003), and school personnel's limited opportunity to 


consider evidence of students’ growth, one might expect 
that conversations about individual students would be 
lengthy enough to be substantive, based on multiple sources 
of quantifiable data, and likely to result in actionable deci¬ 
sions. Unfortunately, in our sample of 25 middle schools, 
this does not seem to be the case. 

On average, teams spent fewer than 2 'A min per individual 
student discussion, even less if the student was an interven¬ 
tion student (just more than VA min). The school with the 
greatest average length of discussion spent an average of just 
more than 714 min per discussion. We frequently observed 
meetings in which student “discussions” consisted mainly of 
one person reading out state test scores and/or attendance 
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data for a long list of at-risk students. It is difficult to imagine 
how 1 x /i min could be an adequate amount of time to identify 
effective strategies to help a child struggling with significant 
reading, attendance, or engagement issues. 

An important question for consideration is why the data 
teams devoted so little time per student in their meetings. 
Perhaps the teams used the meetings to highlight which stu¬ 
dents were struggling, with the expectation that teachers 
would then address those individual concerns on their own, 
in their own classrooms. In cases where a large number of 
students within a school were struggling, perhaps a very 
short period of time is all that could be afforded per student. 
Under such circumstances, it may be more effective for 
teams to accept that not all children can be discussed at 
every meeting, and instead implement a process by which 
specific students are prioritized per meeting. More time 
could be allotted to lengthy discussions of prioritized chil¬ 
dren rather than a very limited amount of time for all chil¬ 
dren (see, for example, Crone, Hawken, & Homer, 2010). 
Prioritization of specific students could rotate from meeting 
to meeting. In addition, schools could hold more frequent 
meetings (weekly, rather than monthly or quarterly) to 
avoid the tendency to use the data team meeting as a non¬ 
productive update of numerous students rather than a data- 
driven discussion leading to actionable decisions for a 
smaller number of children. 

Data Sources 

A review of the data used during observed meetings suggests 
that teams typically adopted a somewhat limited approach in 
their utilization of available data sources. The data presented 
were current and relevant, yet generally limited to a few key 
performance measures, a finding largely at odds with schools’ 
self-reports of how they used data. Most schools self-reported 
using a wide range of data in their DBDM meetings, from 
quantitative measures of academic achievement and progress 
to qualitative measures of family functioning. Yet, many of 
the data sources schools reported using were rarely consid¬ 
ered in observed student (or group) discussions. 

It is important to note there was a difference in how we 
surveyed schools about data use and how we recorded data 
use during observations. In the survey, we asked schools to 
indicate which data sources were used during meetings, 
independent of frequency of use. During observations, we 
summarized the percentage of student (and group) discus¬ 
sions for which each data source was used. This difference 
in data collection could lead to some spurious discrepancies 
between self-reported and observed data use. Even so, we 
believe the limited use of multiple data sources in observed 
meetings is important to discuss. 

In discussions of reading issues, data team participants 
frequently chose to focus on high-stakes state test scores, at 
the expense of examining student performance on specific 


skills and content (e.g., via work samples). This observation 
is in line with our perception that schools were under a great 
deal of pressure to meet annual “adequate yearly progress” 
goals, and that much of the energy of DBDM meetings and 
reading intervention efforts seemed to go toward those stu¬ 
dents who were closest to achieving a score of “meets 
expectations” on the state test. Given this set of pressures 
and incentives, it is understandable, though not necessarily 
effectual, that schools’ primary source of academic data was 
state reading scores. 

In contrast, quantitative data were referenced in only 6% 
of discussions of student behavioral issues. Quantitative 
behavioral data (e.g., number of office discipline referrals, 
attendance) have been established as an important barome¬ 
ter and predictor of student behavioral success or failure 
(Sugai, Sprague, Homer, & Walker, 2000). The observed 
data teams’ relative neglect of such sources of behavioral 
data may be an artifact of the core focus of MS1P (i.e., 
struggling readers). Regardless, it appears that simultane¬ 
ous consideration of quantitative data sources for both aca¬ 
demic and behavioral issues was not commonly tried at 
these middle schools. 

Rather, schools seemed to rely primarily on qualitative 
information when discussing student behavior. This is not 
unexpected, as individuals often try to use contextual or his¬ 
torical information (e.g., family background, peer relations) 
to understand why a student acts a certain way. The discrep¬ 
ancy between reliance on quantitative data for academic 
issues and qualitative data for behavioral issues highlights 
one area in which DBDM practices at these schools could 
be improved. Obtaining a clearer picture of how students 
function across behavioral and academic domains by utiliz¬ 
ing quantitative and qualitative data could help schools 
respond more effectively to the challenges faced by these 
students, based on a more complete picture of each indi¬ 
vidual student’s situation. 

Another important issue for struggling students is atten¬ 
dance. These children are at higher risk of absenteeism, tru¬ 
ancy, and eventual school dropout (Hernandez, 2011). We 
were surprised, therefore, to find that attendance data 
(whether qualitative or quantitative) were used in only 10% 
of discussions, although 80.0% of schools self-reported 
using them in their meetings. 

We recommend that schools utilize more data sources 
in each student’s case, and attempt to use those multiple 
sources of data to triangulate a data-driven definition of 
the problem and proposed solution. We also recommend 
that data teams use a wider range of both quantitative and 
qualitative data sources across multiple contexts, to 
develop a more comprehensive picture of how each stu¬ 
dent is functioning. Furthermore, allotting more time for 
each discussion of a student would encourage team mem¬ 
bers to incorporate more data into their discussions of 
individual students. 
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Actionable Decisions 

Perhaps the most unexpected finding was the infrequency 
with which teams concluded a student or group discussion 
with an actionable decision. The intended functions of 
DBDM teams are to use current and relevant data to identify 
student needs, propose interventions, examine progress, and 
determine whether modifications to existing interventions 
are warranted. It is important to remember that even decid¬ 
ing to maintain a student in his or her current intervention 
was recorded by the observers as a decision made, as was 
identifying a goal, assigning a person, or establishing a time¬ 
line to implement any action. Even with such a low thresh¬ 
old for defining “decision,” we found that data teams actually 
made relatively few actionable decisions: Only 40% of stu¬ 
dent discussions of reading issues and 34% of discussions of 
behavioral concerns resulted in any decision made. 

We speculate some possible reasons for the infrequency 
of actionable decisions. One possibility is that the meeting 
itself was only a starting point for staff to make decisions 
about instructional programming. That is, perhaps the meet¬ 
ings served the purpose of alerting teachers to the needs of 
each struggling student, and teachers later made their own 
decisions about how to help the students in their classrooms. 
Alternatively, perhaps having an opportunity to share con¬ 
cerns about struggling students with colleagues was helpful 
and cathartic to team members, and felt productive and use¬ 
ful, even if very few actionable decisions were made. 

Data teams may have perceived that they had a very lim¬ 
ited range of actions from which to choose. If a middle school 
student continues to struggle after receiving months of inten¬ 
sive instruction in a small group setting, DBDM teams may 
not feel that the school has any options that would be effec¬ 
tive with this student. Without options for interventions, it 
may be difficult to make data-based decisions that will lead 
to improvements in student performance. 

Finally, as noted previously, we observed two to seven 
meetings per school. It is possible that more actionable 
decisions were made at meetings that we were unable to 
observe. However, we believe this is unlikely as the district 
liaisons to the project scheduled observations of data team 
meetings that were representative of each school’s data 
team process. 

Once students were placed into interventions, it appears 
that data teams were also unlikely to follow up on whether 
or not interventions were implemented with fidelity. On the 
SIS, only 16% of the schools indicated that a system for 
collecting data and conducting observations of reading 
intervention fidelity was firmly in place. Another 16% indi¬ 
cated such a system was partially in place. Without consis¬ 
tent attention to actionable decisions and follow-up on 
fidelity to intervention, it is hard to conceive how the actions 
of the DBDM teams could have a significant impact on stu¬ 
dent outcomes. If DBDM teams are to fulfill the potential 


that many experts argue is possible (e.g., Hamilton et al., 
2009; Mandinach, 2012), then attention to critical details of 
the DBDM process must become integrated into the regular 
team routines. 

Based on our findings, we recommend that DBDM 
teams conduct a post-meeting assessment to determine 
whether the team has met key objectives. The assessment 
could be simple and require the team to consider important 
questions, such as the following: (a) What decisions have 
we made that we expect to lead to improved student perfor¬ 
mance? (b) Did we have all of the relevant data we needed 
to make the decisions? (c) Is the process for implementing 
the decisions clear (e.g., person assigned, goal set, timeline 
established), and (d) To what extent did we address the 
needs of the students that we designated as highest priority? 
If data teams were to regularly use such a self-assessment, 
these meetings might begin to more closely resemble the 
iterative process laid out in recommended frameworks of 
DBDM (e.g., Hamilton, 2009; Kekahio & Baker, 2013; 
Mandinach, 2012). Participants would notice whether the 
majority of their meetings centered on conversation rather 
than action, and be prompted to make more decisions. 

Alignment Between Self-Report and 
Observations 

This study incorporates both self-report and standardized 
direct observations of school-based DBDM practices, pro¬ 
viding an opportunity to examine the alignment between 
actual practice and self-report. On a number of key vari¬ 
ables, we found weak alignment between survey responses 
and observation data. For instance, although most schools 
reported having strong systems in place to collect student 
data, our observations suggest that relatively few schools 
actually use the majority of this data on a regular basis. 
Furthermore, although almost half of schools reported hav¬ 
ing a systematic process for decision making, most data 
teams actually made decisions less than 25% of the time. 
Clearly, there is a strong disconnect between what these 
schools felt they had in place at their data meetings and 
what was actually occurring. 

Generalizability of Findings 

This study describes the DBDM practices of 25 middle 
schools in six school districts in the Pacific Northwest. 
Participating schools served a wide range of students in 
terms of academic standing, economic need, ethnicity, and 
English Language Learner (ELL) status. Almost all (25 out 
of 28) of the eligible middle schools in the six districts par¬ 
ticipated. If anything, the fact that these schools had used 
DBDM teams as one component of a multi-component 
intervention for 2 or 3 years might have led us to expect that 



Crone et al. 


91 


the schools’ DBDM practices would be more closely 
aligned with expert recommendations than they would for a 
school without this focus. In fact, we suspect the challenges 
identified in this study may be even more pronounced in 
schools in the early stages of developing a DBDM team 
process. 

We expect that it is not atypical to have a gap between 
what staff believe they are accomplishing within their 
DBDM meeting and what they are actually completing. 
These findings call into question previous studies of self- 
reported DBDM practices and lead us to believe that even 
less is understood about actual DBDM practices than the 
current, slim literature reports (e.g., Feldman & Tung, 2001; 
Marsh et al., 2014). Thus, we argue that additional studies 
of actual observations of DBDM practices are greatly 
needed. 

Limitations 

Newly developed observation tool. The instrument used to 
observe data teams, DTTO 3.0, was developed specifically 
for this study, rather than a published observation protocol 
with established validity and reliability criteria. This was 
necessary as an extensive literature review yielded no other 
published tool adequate for our purposes. We developed the 
tool as an instrument to reflect the structure and process of 
school-based data team meetings, and were able to capture 
the wide variability of practices in use across schools. The 
DTTO went through several iterations, from the pilot year 
to the third year of the study. The purpose of each iteration 
was to make the DTTO more usable, functional, and reflec¬ 
tive of actual practices in schools. IOA was high and con¬ 
sidered strong enough to demonstrate the reliability of these 
findings. However, the psychometric properties of the 
instrument have not been studied in detail, and will need to 
be a focus of future investigation with this tool. Further¬ 
more, in an attempt to create an instrument general enough 
to be used across multiple teams, the specificity of unique, 
individual team practices may have been lost—as in the 
type of detail one might get from an in-depth ethnographic 
or case study approach. Although such an approach could 
provide a more detailed picture of practices within a school, 
it was not feasible for us to provide that level of specificity 
across all 25 schools in this study. 

Observation of DBDM process limited to data team meet¬ 
ings. It is also possible that by focusing our observations on 
only data team meetings, we neglected to document impor¬ 
tant aspects of the full DBDM process in each school. We 
chose to focus our observations on the DBDM team meet¬ 
ing, as we expected this would be the most efficient manner 
to gather the most information about each school’s actual 
DBDM practices. As suggested previously, school person¬ 
nel may have actually used the information shared during a 


team meeting to make individualized instructional deci¬ 
sions and modifications on their own, information not 
included in our analyses. In addition, we were not able to 
observe every single data team meeting that occurred at a 
school. We attempted to observe three meetings per school, 
across the entire school year to obtain a representative sam¬ 
ple of each school’s data team meeting process. However, 
many schools met more frequently than 3 times per year, 
and the data practices from those meetings were also not 
documented or incorporated into our current analyses. 

One aspect of our data team observation tool required 
self-report data by the team leader, that is, interview ques¬ 
tions about the non-observable preparation and follow-up 
elements of the data team process. Given the challenges 
noted with weak alignment between self-report and obser¬ 
vational data, the utility of that information should be con¬ 
sidered with caution. 


Implications for Practice 

Need for a well-specified model for implementation of DBDM 
practices. First, we believe these findings underscore the 
strong need for effective professional development and capac¬ 
ity building in the area of school-based DBDM practices. 
Although schools in the present study had already focused for 
2 to 3 years on DBDM teams as one piece of a multi-compo¬ 
nent intervention, observations of DBDM team meetings 
revealed a number of challenges to effective practices. The 
current literature on school-based DBDM practices concurs 
and calls out the importance of integrating effective DBDM 
practices into the routine school culture (Hamilton, 2009; 
Means etal., 2010), while lamenting the dearth of adequate 
training and support for building these DBDM skills with 
both pre-service and in-service educators (Mandinach, 2012). 

To address this need, a model for the effective implemen¬ 
tation and capacity building of school-based DBDM is 
needed. Such a model could be defined, tested, and modified 
by researchers and educators working in partnership. The 
model should be based on a realistic understanding of 
schools’ needs, abilities, and what can be accomplished given 
current resources and goals. The work described in the cur¬ 
rent article offers a reasonable starting point for building this 
model, as it provides insights into the DBDM practices of 25 
typical middle schools and describes a tool, the DTTO 3.0, 
that could be used to document and compare changes in 
DBDM practices over time. A model for implementation of 
DBDM practices should be specific, including recommended 
time frames for student discussions, timelines for frequency 
of meetings and following up on action items, and a self- 
assessment tool that data teams could use to determine 
whether they had met key objectives during each meeting. A 
well-defined DBDM model would also address capacity 
building, both within the school and the school district. 
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Impact of DBDM practices on student outcomes. A number of 
studies, reports, and practice guides have underscored the 
need for data teams to regularly use effective DBDM prac¬ 
tices in schools (Hamilton, 2009; Kekahio & Baker, 2013; 
Means et al., 2010), especially in connection with students 
struggling academically or behaviorally. The assumption of 
these calls to action is that students will benefit from data 
teams’ effective use of data-driven practices. The literature 
on formative assessment supports the expectation that effec¬ 
tive use of current and relevant data will positively affect 
student achievement (e.g., Fuchs & Fuchs, 1986; Marcotte 
& Hintze, 2009). However, the extent to which data teams 
can and do achieve these goals has yet to be documented. We 
identify a need, not only for additional studies of the actual 
DBDM practices in schools but also to examine the impact 
of those practices on important student outcomes, both aca¬ 
demic and behavioral. If a causal and beneficial link between 
effective data team practices and student outcomes can be 
tested and found, this would provide additional support for 
investing more time, effort, and resources on data team 
DBDM practices and professional development. 
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